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A general result on the method of randomized stopping is proved. It is applied to optimal 
stopping of controlled diffusion processes with unbounded coefficients to reduce it to an optimal 
control problem without stopping. This is motivated by recent results of Krylov on numerical 
solutions to the Bellman equation. 

Keywords: controlled diffusion processes; optimal stopping 

1. Introduction 

It is known that optimal stopping problems for controlled diffusion processes can be 
transformed into optimal control problems by using the method of randomized stopping 
(see [2] and [8]). Since only a few optimal stopping problems can be solved analytically 
(see [13]), one has to resort to numerical approximations of the solution. In such cases, 
one would like to know the rate of convergence of these approximations. Embedding 
optimal stopping problems into the class of stochastic control problems allows one to 
apply numerical methods developed for stochastic control [4] . The price one pays for this 
is the unboundcdness of the reward function, as a function of the control parameter. 

Recently, a major breakthrough has been made in estimating the rate of convergence 
of finite difference approximations for the pay-off functions of stochastic control problems 
(in [9], followed by [10] and [11]). Applying Krylov's methods, new rate of convergence es- 
timates can be found in [1, 5, 6, 7]. New estimates applicable to numerical approximations 
of normalized Bellman equations appear in [12]. 

Our main result. Theorem 2.1, formulates the method of randomized stopping in a 
general setting. Applying it to optimal stopping problems of controlled diffusion processes 
we easily get (see Theorem 3.2) that under general conditions, the pay-off function of 
optimal stopping problem of controlled diffusions equals the pay-off function of the control 
problem obtained by randomized stopping. This result is known from [8] in the case where 
the coefficients of the controlled diffusions are bounded in the control parameter (see 
Section 4 of Chapter 3 in [8]). In Theorem 3.2, the coefficients of the diffusions and the 
functions defining the pay-off may be unbounded functions of the control parameter. Also, 
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they need not satisfy those conditions on continuity which are needed in [8]. Theorem 
2.1 can also be applied to optimal stopping of stochastic processes from a larger classes 
than that of diffusion processes. For the theory of controlled diffusion processes, we refer 
to [8]. 



2. Main result 

Let P) be a probability space equipped with a filtration {Tt)t>Q, such that 

contains all P-nuU sets. Let T denote the set of finite stopping times. Let ^ denote 
the set of all processes F = {Ft)t>o which are {^t}t>o adapted, right-continuous and 
increasing, such that 

Vujen Fo{lu)^0 and lim Ft(tj) = 1. 

t — 'OC 

Let ^ be a class of non-negative adapted locally integrable stochastic processes r = 
irt)t>o such that 

/>OC 

Tt dt = OO. 

Let 9\n denote those stochastic processes from ^ which take values in [0,n]. Set = 

Theorem 2.1. Let {ht)t>o be a progressively measurable process with sample paths con- 
tinuous at 0. Assume that for all t>0, \ht\ < ^ for some random variable ^ satisfying 
< oo . Then, 

/•oo 

supE/i^ = sup E / ht dFt < oo. (2.1) 

rsT Fey Jo 

Theorem 2.1 is will be proven at the end of this section. 
Theorem 2.2. Let {ht)t>o be an adapted cadlag process such that 

E sup \ ht \ < oo . 
t>o 

Then, 

supE / htrte Jo''" "di = supE / htrtc Jo " dt = supE/ir < oo. (2.2) 
reK Ja rem Jq rei 

Proof. Let r eUK. If tpt = e -fo ^""^"^ then 1 — (p Hence 

htrte -^0 '^"'^^ di < sup E / /i( di^t = supE/i^- < oo. 



Fed Jo rei 
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by Theorem 2.1. On the other hand, for r G T, let r" = for f < r and r" = n for t > r. 
Set 



0, 

1 _ e-"(*-'^) 



Then, for any uj £VL and any 5 > Q, 



" 



dt: 



for t <T, 
for t > T. 



ht dF4"==/„ + J„+A'„, 



where 



/„:=/ /Jrdi^,", J„:=/ (/it-/v)di^r, ^« := / /^t dF," 



Notice that as n — > oo, 



/„ = -c-"*) ^/i^ and |if„| <sup|/it|e-(^+^)"->0. 

t>o 



Furthermore, 



I J„| < sup \ht ~ hr\ as (5 0. 

t6[T,T + (5] 

Hence, for any uj E fl, first taking tlic Hmit as n ^ oo and tlicn the limit as 5^0, 



Clearly, 



Hence, by Lcbcsgue's theorem, 



dt 



< sup |/lt| 
t>o 



supE / htrtO 
rem Jo 



lim I 

n — ^oo 



dt 



r/o'^"'i"dt> 

n — *oo J Q 

= E lim / htr]^e~l>''"'^''dt = Ehr. 
Proof of Theorem 2.1. Due to the assumptions on ht in Theorem 2.1, 



sup E / ht dFt < sup E / ^dFt =E^<ao. 
Fed Jo Fes Jo 



Let T G Define 



■= l{r>0}l{t>r} + 1{t=0}(1 - e "*). 



□ 
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Clearly, G" S S". Using the argument of the proof of Theorem 2.2, it is easy to see that 
E/i^= lim E / /i4dGJ'<supE / htdFt. 



Fes Ja 

To complete the proof, one needs to show that 

poo 

supE/i^ >E / htdFt 

rGT Jo 

holds for any £ ^J. To this end, let us define the time-change (3 by 

(3{r) ■.= \TLi{t>Q:Ft>r}. 
It is then easy to see that f3{r) is a stopping time for every r > and that 

/•oc /•! /•! 

M:^E htdFt hp^r)dr= E/i,3(^) dr 

Jo "'0 Jo 

< E / sup E/i^ dr = sup Ehr . 

Jo re% tG'X 

Remark 2.3. Notice that from (2.3), we have the existence of f E [0, 1) such that 



(2.3) 
□ 



: / E/l^(,. 

Jo 



M = / Eh0(,.) dr < E/i^(p) = Ehf 

for the stopping time f = /3(f). If ft, is a piecewise constant process with finitely many 
jumps, then the following lemma provides a way of constructing a stopping time r such 
that M < Ehr holds. 

Lemma 2.4. Let (ft.i)f^i, (k)"=i sequences of random variables adapted to a filtration 
(5i)"=i; where Qi contains all Y-null sets, such that for all i, pi > 0, E\hi \ < oo and 



a.s. 

i=l 

There then exist disjoint sets {Ai)"^-^^, Ai E Gi, U A2 U ■ • • U An = SI such that almost 
surely 

E{pihi +P2h2 + ■ ■ ■+Pnhn\gi) < E(/lil^, + /ijIa^ + " ' ' + hnlA„\Ql)- 

Proof. For n~l, the statement of the lemma is obvious. Assume n = 2 (this will illus- 
trate the general case better). Then, since pihi is Qi measurable, P2 = ^ ^ Pi is also Qi 
measurable and 

/ ■.= Plhi -|-E(p2^2|5l) ^Plhl +P2E(/l2|^?l). 
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Let Ai = {hi > IE(/i2|5i)}. This is a Gi set and 

/ < hiU, + ln\AMh2\Gi) ^HhilA, + h2ln\A^ \Gi)- 

Assume that the lemma holds for n — 1 > 1. Let us prove that it remains true for n. Let 
B = {pi<l}. 

I := E(/liPi + h2P2 H h hnPnlGl) 

= E{lB{hipi + h2P2 + ■■■ + hnPn)\gi)+HlB-^hi\gi) =: h + h. 



Then, 



h = isihipi + E{h2Ps H 1- KpnlGi)) 

, f h2P2 H 1- hnPn 



= 1b hipi + {p2 



-PnW 



\ P2-\ \-P-n 



Ql 



smce p2 



+ Pn > is Ql measurable. Let 

' h2P2 + ■ 



Ai^{hi>E 



P2 



Pn 



nB. 



Since Ai is a Qi set, 



h < hllA,+lBE\^ln\A^ 

= hilA, +E{h'2P2 + --- 



h2P2 



KPn 



P2 H 1- Pn 

-Kp'n\Ql)^ 



where ft-'- = ft-ilsl^jy^^, p[ = — jj'^ — on B andp,^ = ("-^1) ^ on B^ , for 2<i<n. Then, 



i'ii-Bi-n\Ai,Pi - p2+-+p„ 
+ p'^ = l. Apply the inductive hypothesis to {h[Yl^2J (Pi)i=2' (5i)"=2- There are 



then disjoint U A3 U • • ■ U = such that A[ G G^ for i = 2, 



and 



^(h'^p'^ + • • • + h'Jjgi) < Eih'^lA:^ + ■■■ + K,1a'^ IGi). 



Hence, 



Ii<hilA,+lBmn\AAh2l 



- K'i-A'JlGi 



We see that (r2 \ Ai ) n ( A'2 U • • • U J = f7 \ Ai . For 1 < i < n, define A, = Br\A'^r\{n\Ai). 
Such Ai arc disjoint, Gi measurable and 



5"= U Ai U A2 U • • • U A„ = f]. 



Thus, 

h <E{hilA, + h2lA2 + ■ ■ ■ + hnlA^lGl). 
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Finally, 

I = h+h< E(/ii(lAi + IbO + h2lA., + • • ■ + K\a„ \Qi). ° 

Let hit) = X^iLi IcTi-i.Ti] for an increasing sequence of finite random variables 

= To < Ti < • • • < T^v and random variables such that E|^i | < oo for alH = 1, 2, . . . , iV. 
Let {!Ft)t>o be the filtration generated by the process h and the processes l(o,ri] for 

1 = f,2,...,Af. Then, by the above lemma applied to hi :~ ^i, pi F^-- — ^Vi_i and 

E / /i(t)dFt=EV/i,p, <eV/i,1a, =E/i(t), 
•^0 z=i 

for any F (z^, where r = X]i!=i '''i'^Ai: with Aj given by the lemma, is a finite stopping 
time. Hence, we note that the inequality 

hf dFf < sup E/it- 

follows for the general case of progressively measurable processes h satisfying the condi- 
tion of Theorem 2.1, by a simple application of the following lemma from [3]. 

Lemma 2.5. Let {ht)t>o be a B{[0, oo)) x -measurable process and let F £^ such that 

E / \ht\dFt < oo. 



Then, for each integer n>l, there exists a finite sequence of stopping times t^^^ such 
that 



and for 



one has 



U — Tg ^ ^ ^ ^N{n) ^ °" 



JV(ri) 

^i"^=E V")l(r.<")r<")lW' (2-4) 



E/ l/i, - /i^"^|dF, ^0 asn-^oo. 



3. Application to controlled diffusion processes 

We first introduce some notions and notations from the theory of controlled diffusion 
processes from [8]. Fix Ts [0,oo). Let (wtjJ-'t) be a c?'-dimensional Wiener martingale, 
that is, w = {wt)t>o is a Wiener process and Wt — Wg is independent of J-s for all < s < t. 
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Let A be a separable metric space. For every i G [0, T], a; S R"* and a £ A, we are given 
a dx d'-dimensional matrix a^lt^x), a d-dimensional vector /3"(i,x) and real numbers 
c"(<,a:), nt,x) and git, x). 

Assumption 3.1. a,j3,c,f are Borel functions of {a,t,x). The function g is continuous 
in {t,x). There exist an increasing sequence of subsets An of A and positive real constants 
K , Kn and m, m„ such that [J^efi ^ ^ '^^'^ s'^c/i n £N, a £ An, 

\a"it,x) - a"{t,y)\ + \l3"{t,x) - /3"(t,y)| < X„|x - y\, 

\a"{t,x)\ + \f3"{t,x)\ < Kn{l + \x\), (3.1) 
|c"(i,a;)| + \f"{t,x)\ < Kn{l + |x|)™", \g{t,x)\ < K{1 + \x\r 

for all xeW^ andte [0,T]. 

We say that a E 2l„ if a = {at)t>o is a progressively measurable process with values 
in An- Let 21 = IJ^^j^ 2l„. Then, under Assumption 3.1, it is well known that for each 
s e [0, T] , .X e and a e 21, there is a unique solution {xt - t £ [0, T — s]} of 



Xt~x+ / a"" (s + u,Xu) dwu + / (s + u, x„) dit, 



(3.2) 



denoted by a;"'^'^. For s G [0,T], we use the notation T(T — s) for the set of stopping 
times T <T — s. Define 

w(s,a;) = sup sup v"''^{s,x), 
aearei(T-s) 

where 



v'^'^s,x)=El, 



(s + t, xt)c-f* dt + g{s + T, x^)e-'^- 



c"'' (s + r, Sr) dr 



and E"^ means expectation of the expression following it, with x"'*'^ in place of Xt 
everywhere. It is worth noticing that for 

Wn{s,x) := sup sup v°'''^{s,x), 

ae%„ tGI(T-s) 



we have 



Wn{s,x) t w{s,x). 



By Theorem 3.1.8 in [8], Wn{s,x) is bounded from above and below. Hence, w{s,x) is 
bounded from below. However, it can be equal to +oo. 
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Let contain all progressively measurable, locally integrable processes r = {rt)[t>^) 
taking values in [0, n] such that n dt = oo. Let = Ukgn 

Next, we prove a theorem which is known from [8] in the special case when A = An, 
K = Kn, m = TO„ for n>l (see Exercise 3.4.12, via Lemma 3.4.3(b) and Lemma 3.4.5(c)). 
Our proof is a straightforward application of Theorem 2.2. Since Theorem 2.2 is removed 
from the theory of controlled diffusion processes developed in [8] , we do not require that 
cr, 6, c, / and g be continuous in {a,x) and continuous in x, uniformly in a for each t 
(conditions which are needed in [8]). 

Theorem 3.2. Let Assumption 3.1 hold. Then, for all (s,x) S [0,r] x , either both 
w{s,x) and 



Proof. Without loss of generality, we may assume that s = 0. Let r G 5H. For t > T, let 
/"(i, x) ^ 0, c"(t, x) = 0. For fixed (at) G 21, set 





are finite and equal, or they are both infinite. 



/t = r*(t,a;t)e 



for t > 0, 




for t < T, 
for t > T. 



Recall that if r G $H, then rt dt ~ oo. Clearly, ft—0 for t > T, and 




Thus, 






where the last equality comes from integrating by parts. We check that for 
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for each a <E 21, Esupj>Q \ht\<oo holds. Indeed, if a £ 2l,i, then 

Esup|/it|<E sup ([ \r'{s,x,)\e-'^' ds+\g{t,xt)\e- 
t>o te[o,T] V"'o 

<TA'„E sup (l + lsti)"" +T/-CE sup (l + |a;t|)" <oo, 

te[0,T] t6[0,T] 

due to estimates of moments of solutions to SDEs (Theorem 2.5.9 in [8]). Since g = g{t, x) 
is continuous, ht is continuous. Hence, by Theorem 2.2, 

supE / ( / /sds + 5t Vte~/o''"'^"di = supE / /^ds + ^r 



= sup E / fs ds + g-r 

Te%{T) Jo 

because ft ~0 and = for t > T, so nothing can be gained or lost by stopping later. 
Hence, for any a G 2t, 



sup e( / r*{t,xt)e-'^'dt + g{T,Xr)e-'^^\ 

£%(T) [Jo ) 



re%{T) LJo 

= supe| r[n{t,xt)e'^^ +5(i,Xt)rt]e-/o'-"'^"dt 
re<n Ljo 

+ g(r,XT)e-^--r'-"<^"^ 
which proves the theorem. □ 
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